Add CCM (CMTrace) log file format option#6284
Conversation
Assisted-by: Claude:claude-4.8-opus Signed-off-by: Tom Plant <tom.plant@devicie.com>
| FileLogger_MaximumSize_Test(tagState, sizeState); | ||
| } | ||
|
|
||
| TEST_CASE("FileLogger_CCMFormat", "[logging]") |
There was a problem hiding this comment.
Not sure this test and the new imports are necessary, since there isn't one for the existing format. Figured I'd submit just in case
There was a problem hiding this comment.
I think this and more tests should be present if we are writing to a defined format that needs to be parseable. The existing format is just "some text" that happens to have a logical format, but doesn't need to be computer readable.
I would want more unit tests that ensure the specifics of the automatic fields and more extensive coverage on the text portion's escaping.
I would also suggest some mechanism to force the entire E2E run in the pipeline to additionally generate CCM logs that could have a post-run validation applied to ensure compat.
|
I like the idea. I would want to see it in the release notes if we take the change. Do you have any documentation about the format that we could include in the code? I could only find general details about the viewer and an implementation of how to write the logs. Maybe we could have a comment like "The format is ...; see for reference" "@JohnMcPMS" also wrote a custom log viewer exclusively for winget logs that you could use. |
|
Thanks, I'm glad you like it. OneTrace has a useful feature to open a folder and show all the merged logs in one page, which would be confusing if there were multiple copies of the same log. I guess we could log to a subfolder? I took this I spent a while looking and couldn't find any documentation for the format. Ended up decompiling OneTrace to find this: // ConfigMgrSupportCenter.Common, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
// Microsoft.ConfigurationManagement.SupportCenter.Common.LogMessage
using System;
using System.Diagnostics;
using System.Globalization;
using System.Threading;
using Microsoft.ConfigurationManagement.SupportCenter.Common;
internal sealed class LogMessage
{
private const string LogLine = "<![LOG[{0}]LOG]!><time=\"{1:HH:mm:ss}.000+000\" date=\"{1:MM-dd-yyyy}\" component=\"{2}\" context=\"{3}\" type=\"{4}\" thread=\"{5}\" file=\"{6}\">";
public int ContextId { get; }
public string Message { get; }
public LogLevel Severity { get; }
public string SourceFile { get; }
public string SourceName { get; }
public int ThreadId { get; }
public DateTime Time { get; }
public LogMessage(string message, LogLevel severity)
{
Message = message;
Severity = severity;
SourceName = null;
Time = DateTime.Now;
StackTrace stackTrace = new StackTrace();
ThreadId = Thread.CurrentThread.ManagedThreadId;
ContextId = Thread.CurrentContext.ContextID;
if (stackTrace.FrameCount > 2)
{
StackFrame frame = stackTrace.GetFrame(2);
if (!string.IsNullOrEmpty(frame.GetFileName()))
{
SourceFile = string.Format(CultureInfo.InvariantCulture, "{0}:{1}", frame.GetFileName(), frame.GetFileLineNumber());
}
}
else
{
StackFrame frame2 = stackTrace.GetFrame(1);
if (!string.IsNullOrEmpty(frame2.GetFileName()))
{
SourceFile = string.Format(CultureInfo.InvariantCulture, "{0}:{1}", frame2.GetFileName(), frame2.GetFileLineNumber());
}
}
}
public override string ToString()
{
return string.Format(CultureInfo.InvariantCulture, "<![LOG[{0}]LOG]!><time=\"{1:HH:mm:ss}.000+000\" date=\"{1:MM-dd-yyyy}\" component=\"{2}\" context=\"{3}\" type=\"{4}\" thread=\"{5}\" file=\"{6}\">", Message, Time, SourceName, ContextId, (int)Severity, ThreadId, SourceFile);
}
}I saw John's viewer and it looked really useful to me as a dev, but IT admins are usually familiar with CMTrace/OneTrace and may not even have VSCode installed (besides figuring out how to sideload an extension). So I don't think it would help them unfortunately. |
| auto settingsGuard = DeleteUserSettingsFiles(); | ||
| SetSetting(AppInstaller::Settings::Stream::PrimaryUserSettings, R"({ "logging": { "format": "ccm" } })"); | ||
| UserSettingsTest userSettings; | ||
| TestHook::SetUserSettings_Override userSettingsOverride{ userSettings }; |
There was a problem hiding this comment.
This appears to be both writing the actual settings file and overriding it via the hook. You should just use the hook if you can. (Although with a different CCMLogger type you wouldn't even need that.)
| FileLogger_MaximumSize_Test(tagState, sizeState); | ||
| } | ||
|
|
||
| TEST_CASE("FileLogger_CCMFormat", "[logging]") |
There was a problem hiding this comment.
I think this and more tests should be present if we are writing to a defined format that needs to be parseable. The existing format is just "some text" that happens to have a logical format, but doesn't need to be computer readable.
I would want more unit tests that ensure the specifics of the automatic fields and more extensive coverage on the text portion's escaping.
I would also suggest some mechanism to force the entire E2E run in the pipeline to additionally generate CCM logs that could have a post-run validation applied to ensure compat.
| + std::to_string(expectedType) | ||
| + R"(" thread="\d+" file="">)" }; | ||
| REQUIRE(std::regex_search(fileContents, ccmPattern)); | ||
| } |
There was a problem hiding this comment.
It feels like we should be using an XML parser here and actually validating the time/date/thread etc.
| if (Settings::User().Get<Settings::Setting::LoggingFormat>() == LogFileFormat::CCM) | ||
| { | ||
| log = ToCCMLogLine(channel, level, message); | ||
| } | ||
| else | ||
| { | ||
| log = ToLogLine(channel, level, message); | ||
| } |
There was a problem hiding this comment.
I would prefer the CCM logger to be its own type. It should be as easy as deriving from FileLogger and overriding the Write* methods as appropriate. Then use the setting during the initialization to choose between the two options.
| } | ||
| catch (...) {} | ||
|
|
||
| void FileLogger::WriteDirect(Channel, Level, std::string_view message) noexcept try |
There was a problem hiding this comment.
I'm assuming that you would not want to allow actual direct writes to the logs, as they would not have the correct format. If possible, it would likely still be wise to avoid making a string copy like the normal write does.
|
|
||
| void FileLogger::WriteDirect(Channel, Level, std::string_view message) noexcept try | ||
| { | ||
| HandleMaximumFileSize(message); |
There was a problem hiding this comment.
We should understand what it means to have the log file wrap when writing a structured format. The wrapping currently preserves a preamble section (should continue to work correctly), then blindly overwrites as it goes. That would break many of the XML fragments and would require some other strategy to allow maintaining the file size limit (read ahead to erase broken fragment, new file instead of circular buffer, two file "circular buffer" that simply truncates when it switches files, etc.)
| << std::setw(2) << std::setfill('0') << localTime.tm_mday << "-" | ||
| << (1900 + localTime.tm_year) << "\"" | ||
| << " component=\"" << GetChannelName(channel) << "\"" | ||
| << " context=\"\"" |
There was a problem hiding this comment.
Might be an opportunity to include a correlation vector or similar here.
| } | ||
|
|
||
| std::stringstream strstr; | ||
| strstr << "<![LOG[" << message << "]LOG]!>" |
There was a problem hiding this comment.
Is there any mechanism to escape ]LOG]!> (or whatever substring actually indicates the end of the log), or does the format just expect that would never be part of a normal log? I don't want us to have to scan every log string, but it is the correct thing to do. If there is no official escape, we could replace it with some slightly modified version (ex. |LOG|!>).
📖 Description
Add a
logging.formatuser setting that selects the format used when writing log entries to a file. The defaultwingetvalue preserves the existing format, whileccmwrites CCM-compatible lines so WinGet logs can be viewed with common IT admin tools (particularly CMTrace and OneTrace).🔗 References
No issue for this one, but credit to @mjr4077au for the idea while we were chatting with chtulip.
🔍 Validation
Set
logging.formattoccm, then opened someWinGetDev.exelogs with OneTrace and CMTraceOpen Questions
✅ Checklist
📋 Issue Type
Microsoft Reviewers: Open in CodeFlow